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Elastic Storage 



HELD OF THBINVEOTION 

The inveirtioix relates to fhe storage of video content 

BACKGROUND OF THE DSTVENTION 
5 Maay video applicatioi)S ate enabled where video is available at various 

tefiolutioDH and/or qualiti^ in one stream. Methods to accomplish this are loosely referred to 
as scalability techniques. There are three axes on wMoh one can deploy scalability^ The&^t 
is scalability on the time axis, often referred to as temporal scalability. Secondly, fliere is 
scalability on the qualify axis (quantizationX often referred to as sigoal-to-xMsise (SISD^) 

10 scalability or fine-grain scalability. The1faiidax:isisfhetesolatiOAa3ds(iu^^ 

image) oflm refected to as spatial soalabiljiy. In layered codings fbebitstream is divided into 
t«ro or more bitslreams, or layers. Bach layer can be coinbined to &xm a single hlg|i quality 
sigoaL Forexample^ the base layer may pmvidealowerqnaUtyvidw 
enhancement lays provides additional information that can enhance the base layer image. 

IS With (Spatial scalafailityy the base hy^ video may have a loworresolutiotitban fbe ii^ut video 
sequence, in wblch case ihe enhancement layer oaciies information which can zestore the 
resolution of flie base layer to the ii^ut sequence level. 

Typically^ fhese scaled video stceams are stored togettiet in a storage device by 
-016 content provider or service provider* so the quality level of the stored video content is 

20 fixed by the processing which was performed priot to storing the content A user can access 
the storage device or the storage device can download the scaled video streams for display at 
a user device. However, storage problems may occur in the storage device. Forexamplcsa 
U5^ may want to record a new video stream but there may not be enough room in the storage 
device to store the new video stream. In'such situations, there is a need for elastic storage. 

25 The invention allows an efiEective way of reducmg the bitrate, while Uttle resources are 
needed to per&rm the operatioiL 
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SUMMARY OF THE INVENTION 

Tbe jnmrKon ovmomes at le^ part of the defideades deiscribed ^ove by 
pzovidlng a mediod and ^paratas farpiovidiDg elaistio storage by reading an enhanceaieat 
layer out of a storage device aod atGenuadng the eDhanoraneat l^y^ to fbeteby low^ the bit- 
5 lateof tiie enhano^ent layer, thus areatmgmore fipace in tho storage device. 

According to one embodimeat of &e invention^ a meibod and appaiatns fbr 
providuig elastic stoxage of layered video data stoned in a sfoiage ^aratus ate disclosed. 
The stored cnhaiicenieiit layer video data is read out of the storage apparatus. The 
enbancement layer video data is then at least partially decoded ot ultimafely completely 
10 deleted. The decoded enlmcementlayCT video data is attenuated in a linear or 

manner. The attempted enhancoinerttla^er video data is the^ The encoded 

attenuated video data is stored back in the storage apparatus. 

These and other aspects of the invention trill be parent ftom and elucidated with reference 
to the ratbodiments described hereafter^ 

15 

BRIEF DESCRIPnON OP TBQB DRAWINGS 

The invention will now be described, by way of exarnple. wiflx reference to the 
aooompanyhig drawings, wfaereia; 

Figore 1 is a block diagEam of a video cotnpression system according to one 
20 embodiment of ^invention; 

Figure 2 is a block diagram of a video coinpressioaa system according to one 
embodiment of the invention; 

Figure 3 is a block diagram of a video oLooder according to one embodiment 
of the invention; and 

25 Figure 4 is a block diagram of a video conqsression system according to one 

embodiment of the invention. 

DETAILED DESCRIPTION OF THE INVENTION 

Figure 1 iUufitiales a video compression arid storage £r/fitem 100 accordi^ 
30 one onbodmient of the invention. The video compression and storage system lOO comprise, 
among other featares, a layered encoder 102, a storage apparatus 104, a control system 105, a 
variable lenglti decoder 106, an attenua£ar 108 and a variable length coder 1 10. 

An ilbistrative exan:qple of a layered encoder is illustrated in Figure 3, but it 
win be understood that other layered encoders can also be used by the inv(^tion and ttie 
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invention is not limited thereto. The depicted encoding system 300 accomplishes layered 
compression, whereby a portion of the channel is used for providing a bw resolution base 
layer and the remaining portion is used for transmitting edge enhancement information, 
whereby the two signals may be recombined to bring th© system up to hl^ resolution. 
5 The encoder 300 comprises a base encoder 312 and an enhancement encoder 

314. The base encoder is con^iised of a low pass filter and downsampler 320, a motion 
estimator 322, a motion conq>ensator 324, an or&ogOMl tanflfistm (e-g.. Discrete Cosine 
Transfbim PCT)) circuit 330, a quantizer 332, a variable length coder (VLC) 334, a bitrate 
control circuit 335, an inverse quanti^OT 338, an inverse transftrm circuit 340, switches 328, . 

10 344» and an inteipolate and upsample drcdt 3S0. 

AninpntYideobloc&316issplitbyasplitter318andsen^ . 
»coder312 andfhe ehhaneement 6ncod^314. £n tfaebase enooder312, the input block is 
xxqmtted into a low pa^fOtor and downsampler 320, The low pass fate^rediieeatiie 
resobition of fbe video block whichis then fed to demotion estimatOit 322. The motion 

15 estimator 322 processes picture data of each Same a$ an I-^tictore, a P^piotuie, or as a B- 
pictqre. Bach of the pictures of the sequ^tiaQy mtered ftam^ is processed aa one of the I-^ 
or B-pictures in. apre^-set manncar, such as in the sequ^e of], B, P, B, . B, P. That 
is, &e motion estimator 322 leffiana to a pie-set reference fiame in a aeries of pictures stored in 
a fiamememoiy not ilhistrated and detects the rnotion vector of a xri^^ a 

20 small block of 16 pixels by 16 Unes of the fiame bdng encoded by pattern matching (block 
Matching) between fhe ma^-*block and die re&zence finamue &r detecting the motion vector 
of the macro-block* 

hx MPEG, there are four picture prediction modes, that is an intra-coding 
(inlra-ftame coding), a forward predictive coding, a backward predictive coding^ and a bi- , 

25 directional predictive-coding. An I-picture is an inlra-coded picture, a P-pictiire is an intra- 
coded or forward predictive coded or backward predictive coded picture, and a B^>icture is 
aniutra-coded, a forward predictive coded, or a H-direotionBl predictive-coded picture. 

The motion estimator 322 performs foiward prediction on a P-pictur© to detect 
its motion vector, Additionally, the motion estimator 322 performs forward.prediction, 

30 backward prediction, and bi-directional prediction for a B-picture to detect the respective 
motion vectors. In a known manner, the motion estimator 322 searches, in the frame 
memory, for a block of pixels which most resembles ttie current input block of pixels. 
Various search algorithms are known in the art. They are generally based on evaluating the 
mean absolute differmce (MAD) or the mean square OTor (MSB) between the pixds of the 
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4 20.12.2002 
ciuraitixQiUi block and IlKi^ of Ih&c^^ The candidate block having tbe least 

MAD or MSB ifi then selected to be the motiozi-oDxnpensdted prediction block. Its lelattire 
location ivith respect to Ihe location of die cuireut input block is the motion vector. 

Upon receiving fbe prediction mode and the motion vector fiom the motion 
5 estimator 322, the motion compensator 324 may re^ out encoded and already locally 

decoded picture data stored in the frame memory in accordance with the prediction mode and 
flie motion vector and may supply the read-ont data as a prediction picture to arithmetic nnit 
325 and switch 344. The arithmetic unit 325 also receives the input block and calculates the 
difiarence betw^een the izqpnt blodc and the prediction picture fiom flie motion compensator 
10 324. The dififerencevahie is then st^pplied to the DCT circuit 3^^^ 

Ifonly the prediction mode is rec^vedi&om the motion estimator 322, thatis» 
if the prediction mode is the intra-coding mode^ ib» motion compensator 324 may not output 
a prediction picture. In such a situation, the arithmetic unit 325 may not peafonn. die above- 
described proces^^ but instead m^ diiecftly output the hxpm block to the DOT circuit 330. 
IS . TheDCTcmmit330per&nnsDCTpmc^mg(mi]ieGuiputsig^ 

arithmetic unit 33 so as to obtain DCT co^cieofs \fvhich are si^plied to a quantiser 332. 
The quantizer 332 sets a quantizalion step (quantization scale) in aocordance with the data 
stxmga quantity in abu£^ (not illustzated) received as a feedbads: and quantizes the DCT 
coe£Sdeots fiom ibe DCT dxcuit 330 usmg the quantization step» The quantized DCT 
20 coefficients are supplied to the VLC unit 334 along witfi the set quantisation step. 
The imit 334 oonv^s the quantteatlcm coefficients 
quantizer 332 into a variable lengdi code, such as a Hufflnan code^ in accoidance wth the 
quantization step supplied from die quantizer 332. The resulting converted quantizati0n 
coefSoienta are outputCed to a buffer not illustrated. The quantization coefficients and the 
25 quantizalion step are also supplied to an inverse quantizer 338 which dequantizes the 

quantization coefiBcients in accordance TOth the quantization step so as to convert the same to 
DCT coefficients. The DCT coefficients ate supplied to the inverse DCT unit 340 which 
patfomis inverse DCT on the DCfT coefficients. The obtained inverse DCT coeffidente are 
thansappUedtothearidim6tiouiEit348. _ 
30 The aridunetto unit 348 receives the inverse DC!T coefficients fiom the hxverse 

DCT unit 340 and the data fiom the nu>tlon compensator 324 depending on the location of 
swxinh 344. The arithmetic unit 34S sums the signal prediction residuals) fiom die inveise 
DCT unit 340 to the predicted picture fiom the motion compeusator 324 to locally decode the 
origmal pictora However^ if the piedition mode indicates hatra-coding, the ou^ut of the 
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inveiseDCTimit340maybBdireodyfedtotfaefi^ The decoded picture 

obtained by tie arithmetic unit 340 is sent to and stored in flie frame memojy so as to be used 
later as a reference picture for an inter-coded picture, forward predictive coded picture, 
backward predictive coded picture^ or a bi-directional predictivo coded picture* 
5 The enhancement encoder 314 comprises a motion estimator 354, a motion 

coinpensator 356, a DCT circuit 368, a quantizer 370, a VLC unit 372, a bitrate controller 
374, an inverse quantizer 376, an inverse DC3T oiicuit 378, switches 366 and 382, sqbtra^tors 
358 and 364, and addm 380 and 388, In addition, the enhancmieaQt encoder 3 14 may also 
include DC-o£feets 360 and 384, adder 362 and Sttbtractor 386. The operation of many of 

10 these component is similar to the operation of fihoilar contponents in the base encoder 312 
and will not be described in detail. 

Ihe output of the axithmetio unit 340 is also supplied to ttie upsampler 350 
which generally recoostmcts the filtered out resohriion fiom the decoded video stream and 
provides a video data stream having substantially the same resolution as the high-resolution 

IS input. However, because of the mteiing and losses resulting fi^ 

decompression, certain errors are prea^ in the reconstructed atream. The etrors are 
detqcmined in die subtraction unit 358 by subtracting the reconstructed hisih^-resolution 
stream fiom &e original, unmodified hi^ resolution stream. 

20 estimator 354. Tliereoonstnictedhigh-re^lution stream is also provide 

which add$ the output from the inverse DCT 378 Cpostibly modified by liie output of the 
motion compensator 356 deprading on the position of flie switch 382). The oulpvft of the 
adder 388 is sillied to tiie motion estimator 354. As a result, the motion estimation is 
performed on the upscaled base layer plus the enhancement layer instead of the residual 

25 dif&ience between the origmdl hi^-resolution stream and the reconstructed high-resolution 
str^un. This leads to a perceptually better picture quality especially for consumer 
applications which have lower bit rates than ptofe38ional applications. 

Furthermore, a DC-offeet op^ation followed by a clipping operation can be 
Introduced into the enhancement encoder 314, wherein the DC-offset value 360 is added by 

30 ddder 362 to the residual signal output &om &6 subtraction unit 358. This optional DCoffset 
and clipping operation allows flie use of existing standards, e.g., MPEG, for the enhancement 
encoder whore the pixel values are in a predetennined range» e.g., 0. . .255. The residual 
signal is normally concentrated around zero. By adding a DC^offe^ value 360, the 
concentration of samples can be shifted to the middle of the range, e.g.9 128 for S bit video 
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samples. Tbeadvantageofthisadditionis&atthestandardcompo^enisofi^ 
ttwedaiwemettt layer canbei^edaidresultm 

Reimnifigto Figure 1, the base and enhancemeait layers cateatedl)/ the layered 
eiicodfir 102 srestoiedsepa^tely in the storage ^aratus 104. mmlheiMerW a control 

5 system 105 decides Ibat more space is needed iu the storage apparatus 104» an enhanoeiiaent 
layercanbeselected&omtbestoredvideostreanis. WhUeflie control system 105 « 
illustrated as beirigpattome storage appaxatos 104, it be mutetatoodthattte 
eystemlOScanbelocatedelsewhereintiiesysteinlOO. ThensercansdecttheappKipnato 
enbancement layer or the control system 1 05 can select fee enbancemert layer.based m 

10 prevlonsly entered criteria. The selected enhancement layer is «ado« of the storage 

■ ^aratiia md sent to the variable leogttl decoder 106. 

The variable length decoder 106 partially decodes Ibe selected enhanoament 
layer. For example, the vari^te length decoder my decode fee DCTcoefficienls (AC and 
DQoftheeelectedenhancemeHtlayer. in this embodiment ofthe invention, fee decoded 

15 DCTcoefficientpareattennatedbyaprBdeterininedconstaiitvalnemfeBatterai^ 

(mttKiplierunit)l08. iiieattennalion will have the effect ofrcdudng fee video resolution of 
Itoenhanoemenllayerandtedacesthebit^ofliieeifeancemeitfla^^ Theattetinated 
enhancement layer is feenie-encodedby the variable lengfeencoder 110 and the redded 
aihancemBnt layer is stoiedbacS: in the storage ^aratiis 104. 

20 If aDC-ofBet andcBpptagopeiationliasbeenpcr&imfld dnringfee creation 

of the stored enhancement btyer video data, as Ulustrated in Figora 3 by el^ents 360 and 
362, the DC-of&et needs to be removed fSrom fee decoded enhancement layer video data 
jttiortoperfiiimingtheattemiationstepinfeeattenuatorlOS, in feis embodiment as 
iIliwtratedinFigiire2,feecQrrespoiidmgDC-offletvalue 109 is removed fiom the DC DCT 

25 coeffioiant(firstcoefficieot)offeedecodedenhancementvideo data by a modification 
(subtraction) mnt 111 prior to being supplied to fee attenuator 108. After fee attenuation 
Btep.feeDC^iSetval«eisaddfldbadkintofeeDCDCTcoefficie«toftheattemjat£d 

enhancement layer video databy amodificaiion (addition) wnlt 113 prior to being sm>pUed to 

fee variable lengfe ^)cx)der. 
3Q Kgnre 4 illustrates a video compression and storage system 200 according to 

anofeer embodiment of fee invention. The system 200 is similar to fee system 100 illustrBted 
inFigorel andliketeferencenomeralahavebeenusedforlikeelements. Infeis 
embodiment, fee attenuator 202 is comprised of a weighting means 204 and a quantizer 206. 
As in Figure 1, the layered encoder 102 produces a base layer video stream and an 
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enhancement layer video stream which are stored in flxe storage apparatp 104. Whea the 
usOT or tho control system 105 selects an enhancement layer for reduction, the selected 
enhancement layer is read out of the storage apparatus 1 04 and partially decoded by the 
variable lengQi decoder 105. The attenuator 202 performs a weighting step and a 
5 qi1?mtig;frti<^n gt^ t>^ft decoHed npT coeffioieHls of Ab enhancement layer. The weighting 
step i$ performed by mult^lying a 8*8 weighting matrix to blocks of DOT coefficients, each 
DOT co&£^ent being thus multiplied by a weighting factor contained in the matrix. The 
result of the multiplication is rounded to the nearest integer, wherdn tho weighting matrix is 
filled with values which amplitude are bettv'een 0 and 1 > set fi>r escan^le to non-uniform 

10 values dose to 1 for low Sequential values and close to 0 for hi]^ frequential valueSj or to 
unifotm values so that all coeSBfiiessits in the 8*^8 DCT block are equally attenuate hi other 
words, Mgher frequency coefiSdents are more attenuated than low frequency coefficients. 
The weighted DCT coelBSeifinta are Hien quantized by dividing the wei^ted DCT 
coefBoients by a quantizalion &ctor for produdng quantized DCT coefficients. The 

15 quantized DCT coefBdents are thCT re-encoded by the v^ 

badcin the storage ^paratus 104. In this enibodiment, while the bit-r^e of the enhancemm^ . 
layer is reduced, emar propagation will occur and the coding-eBiaienoy of the leduoed 
enhancement layer will be reduced. 

It should be noted ibat the above-mentioned embodiments illuatrata tafher than , 

20 limit the invention, and that those sldlled in the art will be able to design many alternative 
embodhnents wifiiout depardng dOtom the scope of the appended daims. In ttie claims^ any 
xeference signs placed between parentheses shall not be constmed as limiting the claim. Tho 
word 'contprifiing' does not e^cchide the presence of other elements or steps than those listed 
in a claim. The invention can be implemented by means of hardware comprising several 

25 distinct elements, and by means of a suitably progranmied computer. In a device claim 
enxmtemtlng several means, several of these means can be embodied by one and the same 
item of hardware. The mere fact that certain measures are recited in mutually diflEferent 
dependent claims does not indicate that a combination of these measures cannot be used to 
advantoge. 
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CLAIMS: 



1. A method for provldiog elastic storage of layered video data stored in a 
s&urage apparatus, contpiisipg the steps o£ 

reading stared ^ihanceEnerit layer video data Old: of ss^ 
at least panially decodiiig the ^ihancemetit layer video data; 
5 attenuating &fi decoded eoh^ 

encoding the attenuated enlxanceznent layer video data; 

storing fho encoded attetmated video date in said storage i^paratus. 

2. The method according to claim 1 > wherein the attenuation reduces the bit-rate 
10 ofthe video data. 

3 . The method according to claim 1^ whezdn DCT coefficients of Ihe decoded 
enhancement layer video data are attenuated. 

IS 4. The rncOcd according to claim 3, wherdn fibe DCT coefScients are attenuated 

by apredetiaiiuned constant value. 

5. Hie mefiiod accordlngto claim 3» wherein the DCT coeffioienib are attemiated 
in anon-Hnear miamier. 

20 

6. The meOiod according to claim 4, whezetD each DCT coefficiait is 
by a wdgbtii^ &ctoi' in a weighting toalrix. 

7. The method accotding to claim 6, wh^rdn Mgher fi:eqaency coefificients axe 
25 more attenuated than low fieqiuencycoefSdents. 



8. The method according to claim 6, wherein the weighted DCT ooefScients are 

quantized by dividing the weighted DCT coeffident? by a quantization ^ctor prior to being 
re-encoded. 
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9. Ttemefliodaoooiding to claim l,fiu:fliercon^ 

removing a DO-offiet value from a DC DC?r coefficient of the decoded 
eidianceinent layer video data prior to &c attenuatioa 6tcp; aiul 
5 adding the DC-oflfeet value back ioto the DC DCT coefficient of the attenuated 

enhancement layer video data before the ^Koding step. 

10. An apparatus G>t providing elastic storage of layered video data $toted in a 
storage ^paratus, comprising: 

10 meaiis for reading stored eDhancement layer Video data ou^ 

q>paxattts; 

decoding meanfi fi>r at least partially decoding Hxe enhanceniBnt layer video 

data; 

attmuation means fbr atCemiaijng the decoded eashanoexnent l^yer video data; 
1^ encoding means fbr encoding the attenuated enhancement layer video data; 

meana fi>r storing flie eacoded attemjated video data m said storage apparatus. 

1 1. The apparatus according to claim 10, wherein the attenuation reduces the bit- 
rate of the video data. 

20 

12. The apparatus according to claim 10, wherein DCT coefficients of the decoded 
enhancement layer video data are atteimated. 

13. The apparatus according to claim 12^ wherein the DCT coefScieats are 
25 attenuated by a predetermined constant value. 

14. The apparatus according to claim 12, wherein the DCT. coefficients are 
attenuated in a non-linear manner. 

30 15. The apparatus according to claim 13, finther comprising: 

weighting means for multiplying eadi coefficient by a w^gbting &ctor hi a 
weighting matrix. 
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16. The flppatatos according to claim 10, whex^ hig^ fiequenoy coeffidente 
W more atcenmted than low fiequency coefSoi^sts. 

17. The ^paraiw according to claim IS, farther comprlshi^ 

5 a quantizer fbr quantiztng tlie vdghted DCT coefiBciezits by dividing the 

we^hted DCT coefiSdents by a quantization factor prior to beiog re-encoded. 

18. The apparatus according to claim 10, fluther comprising; 

means S>r removing a DO-oJEfeet value from a DC DCT coe££icient of the 
10 decoded enhancement layer video data prior to the attenuation step; and 

means fi>r adding the DC^^Tset vahje back into the DC DCT coefficient of the. 
attenuated enhancement layer video data befixre (he encoding step. 
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ABSTRACT: 



Ametbod and appaiatos &r providing elastic sfniage of layered video da^ 
stored in a storage apparatus are disolosed. The Murod wbsxio^Gnt layer video data is tead 
out of fha storage apparatas. The ^anoemeot layer video data is then at least partially 
decoded or ultimately cotx^letely ddeted. The decoded enhancement layer video data i3 
5 attennated in a HnearM non-linear manner. The attenuated enhancomentlayeir video data is 
fhen encoded. The encoded attenuated video data Is stored bac^ in the stonagea^a^^ . 

Fi&l 
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